I too have had a lot of "fun" dealing with this kind of situation, and honestly, I wish the selection formula could be profiled much like the column formulas are.
Still, I know the selection formula can cause something of a drain on performance. The column formula does, too.
Cool idea though!
One other thought -- I've also done things like this with folders. I dunno, it works to a point, but it can also be slow-ish. I may try this out if something else breaks.